Recently, according to upgrading of information infrastructure and the progress in multimedia technology, an AV synchronization system, which reads an AV stream through various transmission lines and media, and reproduces it, is desired and becomes commercial.
Some systems uses a stable line such as a digital broadcast, and other systems read a stream through an unstable line, where a transmission rate varies according to the degree of line congestions, such as internet and wireless LAN.
In addition, in a home PC (personal computer), software capable of editing streams becomes commercial, and various stream data can be created personally.
In a method for synchronization in a conventional AV synchronization reproduction apparatus, when processing a plurality kinds of data such as audio data, video data, in parallel, one process corresponding to one kind of the data serves as the master, and its output timing is used as reference timing so that the other kinds of data are provided with synchronized with the reference timing.
Since audio stop causes a feeling that something is wrong, in order to synchronize between audio and video, an audio processing serves as the master, and a system clock is corrected based on timing information of audio taken from the stream, as disclosed in Japanese Laid-Opened Publication KOKAI No. HEI 7-303240, for example. In video reproduction, the system clock is compared with the timing information taken from the video stream. In order to perform synchronization, when the video leads, video reproduction stops to adjust timing, on the other hand, when video delays, video reproduction accelerates.
FIG. 18 is a chart showing a flow of conventional synchronization processing in the case that a stream containing audio, video, text and a still-image is reproduced based on the audio master. In FIG. 18, numerals 201, 202, 203, 204, and 205 represent a clock process, an audio process, a video process, a text process, and a still-image process, respectively. Thick lines represent the clock process, and the audio process 202 as the master.
In the case of the audio master, timing information PTS (Presentation Time Stamps) is taken from a stream to reproduce, and audio is provided based on the timing of a clock 201 subjected to correction 206. The others, the video process 203, the text process 204, and the still picture process 205 are performed based on this clock as reference 207. For example, in the case of the video process 203, its timing information in a stream is taken from the video stream, and the video is provided at the same time with the referenced clock 201 to synchronize. An upward arrow directing toward the clock represents correction of clock. A downward arrow drawn from the clock represents reference of clock.
FIG. 19 is a chart showing operation in a conventional system in the case that the audio data stops in the audio master. After that, since a conventional system remains the audio master, the timing information PTS cannot be taken from the audio. As a result, the clock cannot be corrected, therefore, the video process, the text process, and the still picture process are continued being performed without being synchronized after the stop.
In a conventional method, since the master is consistently fixed, reference time cannot be obtained, when the audio data stops caused by trouble of a transmission line, or when the audio data is exhausted at a midpoint during edition of a stream, in reproduction in the case of the audio master, for example.
In addition, when an error of clock is extremely large, correction causes a large image skip, sound skip, or the like.
Additionally, even when an error of clock is small, since the correction and reference are consistently performed, this causes additional overhead.